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Abstract 

A piloted simulation study was conducted at the 
National Aeronautics and Space Administration 
(NASA) Langley Research Center (LaRC) to 
evaluate the ability to safely conduct surface 
trajectory-based operations (STBO) by assessing the 
impact of providing traffic intent information, 
conflict detection and resolution (CD&R) system 
capability, and the display of STBO guidance to the 
flight crew on both head-down and head-up displays 
(HUD). Nominal and off-nominal conflict scenarios 
were conducted using 12 airline crews operating in a 
simulated Memphis International Airport terminal 
environment. The flight crews met their required 
time-of-arrival at route end within 10 seconds on 98 
percent of the trials, well within the acceptable 
performance bounds of 15 seconds. Traffic intent 
information was found to be useful in determining 
the intent of conflict traffic, with graphical 
presentation preferred. The CD&R system was only 
minimally effective during STBO because the 
prevailing visibility was sufficient for visual 
detection of incurring traffic. Overall, the pilots 
indicated STBO increased general situation 
awareness but also negatively impacted workload, 
reduced the ability to watch for other traffic, and 
increased head-down time. 

Introduction 

The Next Generation Air Transportation System 
(NextGen) vision transforms the air transportation 
system to meet the projected growth in aircraft 
operations expected in the 2025 time-frame in a safe, 
efficient, and reliable manner. Emerging NextGen 
operational concepts [1], such as trajectory-based 
airborne and surface operations, equivalent visual 
operations, and high-density arrival and departure 
operations, represent a revolutionary approach to air 
traffic management. A net-centric infrastructure is 


envisioned, enabling a safe and efficient airport 
surface environment by providing the information 
necessary to improve flight deck and ground vehicle 
situational awareness through display of traffic, 
airport moving maps (AMM), and proactive alerts of 
runway incursions and surface traffic conflicts. 

Surface Trajectory-Based Operations (STBO) is 
a NextGen operational concept aimed at improving 
the efficiency and safety of surface operations at 
high-density airports. Surface movement 
management tools will utilize improved surveillance, 
environmental and operational conditions, and other 
information to sequence aircraft for departure, issue 
conflict-free time-based taxi routes, monitor taxi 
conformance, and revise taxi plans as necessary [2, 
3]. The taxi routes are intended to be transmitted 
through data communications protocols and 
displayed to the flight crew on a flight deck surface 
movement display [4-8]. 

Although STBO is intended to eliminate surface 
traffic conflicts, on-board conflict detection and 
resolution (CD&R) systems will provide an 
additional, protective safety layer for NextGen 
operations in the event that tactical or strategic 
situation awareness is not sufficient or human errors 
or blunders occur. The National Aeronautics and 
Space Administration (NASA) has developed a 
CD&R concept that leverages advances in flight deck 
displays and technologies to promote enhanced 
surface and airborne traffic awareness with 
associated flight deck alerting concepts for safety 
assurance [9-13]. These concepts employ continual 
ownship and traffic data monitoring and algorithms 
to detect potential conflicts on the runway and during 
taxi operations for aircraft and surface vehicles. 
NASA has also conducted initial research on ground- 
based CD&R for air traffic control (ATC) STBO 
decision support tools [14, 15] and the integration of 
ground- and flight deck-based CD&R for STBO [16]. 



A high-fidelity full-motion piloted simulation 
study was conducted to evaluate the ability to safely 
conduct STBO by assessing the impact of providing 
traffic intent information, flight deck-based CD&R 
system capability, and the display of STBO guidance 
to the flight crew on both head-down and head-up 
displays (HUD). This paper describes the test set-up, 
method, and quantitative and qualitative results. 

System Description 

Simulator 

This research was conducted in the Research 
Flight Deck (RFD) simulator at NASA Langley 
Research Center (LaRC) (Figure 1) which is a high- 
fidelity, six degrees-of-freedom motion-based large 
commercial aircraft simulator with full-mission 
capability and advanced glass flight deck displays. 
Operations were conducted at the Memphis 
International (KMEM) airport. The out-the-window 
(OTW) scene included realistic taxiways and 
runways with appropriate markings, airport lighting, 
and other aircraft in simulated visibility conditions 
and provided approximately 20/40 visual acuity with 
a collimated 200° horizontal by 40° vertical field of 
view at 26 pixels per degree resolution. 

The RFD is equipped with dual 46° horizontal x 
34.5° vertical field-of-view commercial HUDs; 
however, only the HUD located on the left or 
captain’s side was utilized. 



Figure 1. RFD Flight Deck 

As shown in Figure 1, the simulator had four 
large main instrument panel displays referred to as: 
(left to right) pilot’s Primary Flight Display (PFD), 


pilot’s Navigation Display (ND), co-pilot’s ND, and 
co-pilot’s PFD. Each display panel had 13.25 inch x 
10.5 inch viewable area at 1280 x 1024 resolution. 

Two Electronic Flight Bags (EFBs) were 
installed. Each provided a display resolution of 1024 
x 768 pixels over a 10.4 inch diagonal area. The 
EFBs were mounted outboard of the PFDs and were 
located within the pilot’s primary field-of-view as per 
the Advisory Circular (AC) 25-1 1 A [17], 

Flight Deck Displays 

The PFD, ND, and HUD formats mimicked 
current state-of-the-art production aircraft. Additions 
were made to accommodate study objectives. 

Primary Flight Display 

The PFD included an ATC message area on the 
outboard portion of the panel showing incoming and 
outgoing ATC data-link communications in textual 
format. Incoming messages were color-coded green 
while outgoing messages were white. All messages 
were time-stamped. 

Navigation Display 

The ND was split, showing a half-screen map 
display and half-screen Engine Indication and 
Caution Alerting System. An AMM was not shown 
on the ND. 

Head-Up Display 

Standard on-ground HUD symbology was used. 
The ground speed was displayed in the lower left. 
The STBO advised speed was shown above the 
ground speed. 

Electronic Flight Bag Display 

The EFB was used for communications, charts, 
checklists, and AMM through either a bezel button or 
touch screen interface. 

The communications function was used to 
display and respond to ATC data-link messages. An 
incoming ATC message was displayed in green on 
the EFB and also on the PFD. Once the message was 
acknowledged, the message color changed to white 
on the EFB and an acknowledgement message 
(‘WILCO’) was displayed on the PFD. 

The charts function provided the ability to 
display arrival, approach, departure, and airport 



charts. Pan/zoom (full page, half of the diagram, one 
fourth of the diagram) capability was provided. 

The checklists function provided the capability 
to display all aircraft checklists. 

The EFB was the only location used to display 
the AMM. Range control for the AMM was 
provisioned at 0.4 nautical miles (nm), 0.8 nm, 1.6 
nm, and 3.2 mn. The basic AMM display included an 
airport layout showing runways, taxiways, and 
buildings. Surface (tan) and airborne (cyan) traffic 
icons were shown along with the ownship position 
(white chevron). The AMM used a KMEM airport 
geographic database developed to RTCA standards 
[18]. In this document, all figures of the AMM are 
shown without the surrounding EFB bezel button 
structure. Specific AMM configurations used for the 
study are provided in the Test Method section below. 

Air Traffic Control Simulation 

ATC instructions and other aircraft requests and 
replies were simulated via a speech generation 
system to increase the fidelity of the simulation and 
provide normal pilot workload demands. The 
messages were played through the intercom or 
through the flight deck speaker system when the 
ownship and simulated traffic reached specified 
locations or timings to coincide with the scenario 
task. The subject pilots were asked to provide radio 
replies as per normal operating procedures. Human 
back-up was present if additional or unscripted 
communication or clarification was necessary. 

Conflict Detection and Resolution 

A CD&R system was active during the testing. 
This system was developed based on Safety, 
Performance, and Interoperability Requirements 
(SPR) for Enhanced Traffic Situational Awareness on 
the Airport Surface with Indications and Alerts 
(SURF IA) specifications [19] as well as NASA 
CD&R research. 

NASA CD&R Research 

The NASA taxi conflict monitor was designed 
to detect and alert for ground taxi conflicts in the 
airport movement area by computing distances 
between the ownship and traffic, closing speeds, time 
to closest point of approach and other parameters to 


determine if criteria and thresholds are met for 
issuing alerts [20]. 

Indications and Alerts 

Indications and alerts (IA) notify the flight crew 
of potentially hazardous situations. IA specifications 
are defined in [19]. This study focused on STBO taxi 
operations and only included taxi conflicts; therefore, 
indications were not generated and, consequently, 
will not be discussed further in this document. 

Alerts identified potential collision hazards 
which require immediate flight crew awareness and 
may require timely action or response. A two-level 
alerting scheme was utilized. 


Figure 2. AMM Showing Caution Alert 

Caution alerts were generated for conditions 
that required immediate flight crew awareness and 
subsequent flight crew response. A caution alert was 
displayed on the AMM as an enlarged yellow traffic 
symbol surrounded by a yellow circle for the relevant 
traffic, an identification tag and ground speed in 
knots, and a yellow line around the relevant runway, 
if applicable. An alert message (“Caution, Traffic”) 
was displayed at the bottom of the AMM in yellow 
text along with the estimated distance to the traffic. 
An audible annunciation was also made (“Caution, 
Traffic, Caution, Traffic”) (Figure 2). 







Figure 3. AMM Showing Warning Alert 

Warning alerts were issued for conditions that 
required immediate flight crew awareness and 
immediate flight crew response. Warning alerts could 
occur without preceding caution alerts. A warning 
alert was displayed in the same manner on the AMM 
as a caution alert, except the warning was associated 
with the color red, a square was used to surround the 
traffic symbol, and the alert message was “Warning, 
Traffic” (Figure 3). 

Surveillance Data 

The quality and accuracy of reported traffic 
surveillance data are critical to the integrity of the 
AMM traffic displays and the CD&R capability. The 
traffic position accuracy was simulated as dependent 
upon the Global Positioning System (GPS) 
measurement errors. A Gauss-Markov process 
modeled the time correlation between successive 
position measurement errors [21]. It was assumed 
that ADS-B would be used as the means for 
transmitting (ADS-B Out) and receiving (ADS-B In) 
these GPS-based aircraft surveillance data. Although 
ADS-B transmission qualities and effects were not 
modeled for this study, the traffic positional data was 
updated at one hertz to simulate ADS-B transmission 
rates. Latency effects, transmission line-of-sight, 
bandwidth blockage, and vertical position accuracy 
were not modeled. 


The Navigation Accuracy Category for Position 
(NACp) describes the accuracy of positional 
information. NACp values range from 0 to 1 1 [22]. 
The NACp categories of 8 and higher are listed in 
Table 1 with their associated horizontal Estimated 
Position Uncertainty (EPU) values. 


Table 1. NACp Horizontal Accuracy Bound 


NACp 

95% Horizontal Accuracy - Estimated 
Position Uncertainty (EPU) 

8 

EPU < 92.6 m (0.05 nm, 305.6 ft) 

9 

EPU < 30 m (99 ft) 

10 

EPU < 10 m (33 ft) 

11 

EPU < 3 m (9.9 ft) 


Both DO-289 (Minimum Aviation System 
Performance Standards for Aircraft Surveillance 
Applications) [23] and DO-322 (SPR for Airborne 
Traffic Situational Awareness for Surface Operations 
(ATS A- SURF) Application) [24] define the 

minimum requirement for horizontal position 
accuracy for depiction of ground traffic on an AMM 
to be at least 30 meters (m) within 95% containment 
bounds, equivalent to NACp 9. Requirements for 
other navigation parameters are also specified in 
these reports but will not be addressed in this paper. 

The SURF IA SPR has proposed horizontal 
position accuracy requirements [19] for the SURF IA 
function. Through analysis, the SPR identified that to 
meet safety requirements, horizontal position 
accuracy when on the airport surface needs to be at 
least 10 m within 95% containment bounds (NACp 
10) to allow alerts to be issued for traffic at virtually 
all airports in the National Airspace System. 

Traffic positioning accuracies equivalent to 
NACp 9, 10, and 11 were used. Truth data, with no 
accuracy errors, was also recorded. The accuracy 
levels (e.g., NACp 9, NACp 10, etc.) were controlled 
for individual traffic. 

STBO Guidance Algorithm 

For all ownship routes, a nominal taxi trajectory 
was created. The trajectory was built using the 
ownship ’s performance and loading, assuming 
relatively low visibility (1,800 feet (ft)), and planning 
a nominal acceleration and deceleration at the start, 
end, and around turns to create a planned, distance, 



time, and speed taxi profile. The profile assumed 15 
knots (kts) in straight taxi segments and a minimum 
of 5 kts through the turns. The designed trajectories 
were flight-checked for reasonableness by several 
check-pilots. 

STBO guidance information was computed 
using a feedback control law with actual performance 
compared against these desired trajectories to provide 
an advised ground speed (GS). The control law used 
intermediate waypoints along the route to implicitly 
obtain STBO compliance along the route as well as 
the final Required Time-of-Arrival (RTA). The 
intermediate waypoint performance was not 
explicitly shown nor instructed to the crews, but it 
was part of the control law in computing the advised 
speed for the crews. 

The advised speed was limited to maximum and 
minimum values: 30 kts and 10 kts, respectively, on 
straight-aways and 10 kts and 4 kts in turns. 

The crew was instructed that the advised GS 
(displayed in green at top left of AMM) was to be 
used as a reference for reaching the RTA point (green 
diamond near end of route) at the RTA (displayed in 
green at bottom of AMM display) (Figure 4). It was 
not required that pilots track the advised GS 
precisely. Acceptable performance was met if the 
time-of-arrival at the RTA end point was within +/- 
15 seconds of the RTA. The Estimated Time-of- 
Arrival (ETA) and seconds early or late was 
displayed at the bottom of the AMM in white (Figure 
4) to give the flight crew additional information on 
how well they were progressing along the route. If 
the crew were more than 15 seconds early or late, the 
words ‘EARLY’ or ‘LATE’ would also be displayed 
at the bottom of the AMM. 

Test Method 

The test objectives were to evaluate the ability to 
safely conduct STBO by assessing the impact of 
providing traffic intent information, CD&R system 
capability, and the display of STBO guidance to the 
flight crew on both head-down and head-up displays. 
Evaluations were conducted during STBO taxi 
operations from the ramp to the departure runway 
utilizing two HUD conditions and two AMM 
conditions with traffic transmitting various levels of 
horizontal position accuracy. 


Evaluation Pilots 

Twenty-four commercial pilots participated. The 
test subjects were paired by airline and role (Captain, 
First Officer) to ensure crew coordination and 
cohesion with regard to terminal and surface standard 
operational procedures. All pilots held an Airline 
Transport Pilot rating. The Captains had an average 
of over 17,000 flight hours with 25 years of 
commercial experience. The First Officers had an 
average of over 13,000 flight hours with 20 years of 
commercial experience. 

Display Conditions 

Symbology was added to the basic AMM format 
and to the HUD to enable the flight crew to 
efficiently and safely conduct STBO taxi operations. 
Two AMM and two HUD display conditions were 
evaluated as described below. 

The Map C condition consisted of a basic AMM 
format and added the display of textual STBO 
guidance information, selected traffic information, 
and the ownship cleared taxi route as a magenta line. 

The crew had the capability to obtain selected 
traffic information (aircraft identification; bearing, 
heading, and range from ownship; ground speed; and 
cleared taxi route) by touching the traffic icon 
symbol on the AMM. By reading the traffic’s cleared 
taxi route, the crew could figure out route conflicts 
with the ownship ’s cleared taxi route. 

The Map D condition was created by the 
addition of graphical ownship and traffic intent and 
graphical traffic route (Figure 4). 

In this condition, when selected traffic 
information was selected, the traffic’s cleared route 
was also displayed graphically in tan (Figure 4). 

Graphical intent information was also displayed 
for both the ownship and selected traffic using solid 
and open circles. The solid traffic intent symbol gave 
an indication of position required in 30 seconds to 
achieve the RTA. The open trend symbol gave an 
indication of the position of the aircraft in 30 seconds 
based on the planned speed required to meet the 
RTA. When the trend symbol surrounded the intent 
symbol, as shown in the left AMM image in Figure 4, 
the aircraft was precisely tracking the STBO 
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Figure 4, STBO Guidance Information Displayed on AMM 


clearance. If the trend symbol was displayed behind 
the intent symbol, as shown in the right AMM image 
in Figure 4, the aircraft was behind schedule and may 
reach the RTA point late. If the trend symbol was 
displayed ahead of the intent symbol, the aircraft was 
ahead of schedule and may reach the RTA point 
early. 

The graphical taxi routes and intent and STBO 
status information provided the capability to view the 
route conflicts at-a-glance rather than having to 
interpret the textual clearance in the information area 
to determine whether there was a potential conflict. 
By viewing the traffic’s intent and trend symbols, it 
was obvious when a collision was predicted; the 
ownship and traffic intent and trend symbols overlaid 
each other. 

The F1UD was used during a portion of the 
STBO taxi scenarios. The standard F1UD surface 
symbology (described above) was used with the 
addition of the advised GS (displayed above the GS). 
This allowed the Captain to remain heads-up to view 
the STBO guidance. When test conditions did not 
require use of the F1UD, the F1UD was stowed so it 
would not be an influence or obstruction. 


Test Matrix 

A total of six trials (four nominal, two off- 
nominal) were conducted with each crew. All trials 
were conducted in daytime 1,800 ft visibility. Traffic 
was transmitting position accuracy ranges from 
NACp 9 to NACp 11; therefore, all the airport traffic 
was displayed on the AMM. 

Nominal trials were conducted to evaluate the 
flight crew’s performance in following STBO taxi 
guidance using F1UD and AMM symbology. Two 
AMM conditions (Map C and Map D) were 
evaluated between subjects, using two F1UD display 
conditions (F1UD or no F1UD) during departure taxi 
scenarios, for a total of four nominal trials. 

Off-nominal trials were conducted between 
subjects to evaluate collision avoidance capability 
during STBO. The focus of the off-nominal scenarios 
was to detennine the usefulness of traffic intent 
information for taxi collision avoidance and to 
determine the usefulness of CD&R for collision 
avoidance during STBO. Each crew completed two 
off-nominal trials as Test Runs 3 and 6 of six trials. 
The F1UD was not utilized during the off-nominal 
trials. A detailed description of the off-nominal 








scenarios is given below. Each crew evaluated these 
two scenarios using either the Map C or D condition 
with conflict traffic transmitting either NACp 9 or 
NACp 10 position accuracy; therefore, three data 
points were collected for each combination of 
conditions. Alerts were not issued for traffic 
transmitting NACp 9 position accuracy, but were 
issued for traffic transmitting NACp 10 position 
accuracy (as per DO-323) [19]. 

Conflict Scenarios 

Two conflict scenarios were utilized. Every 
effort was made to produce similar timings; however, 
a certain amount of variability was naturally 
introduced due to the maneuvering conducted by the 
pilot (e.g., taxi speed). 

Taxi Head-On Conflict Scenario 

This scenario tested a potential head-on collision 
if no action were taken because the aircraft were 
given conflicting STBO taxi clearances. 



Figure 5. Taxi Head-On Conflict Scenario 

An approach and departure flow was simulated. 
Traffic was approaching Runway 18R (Figure 5, 
orange route) and Runway 18C (Figure 5, orange 
dashed route), spaced five nm apart and staggered 
between runways. This traffic landed, exited the 
runway, and taxied to the terminal. There was a 
departure flow using Runway 18L via the magenta 
and blue colored routes shown in Figure 5. 

At the beginning of this scenario, the ownship 
was parked on the ramp near Geographic Position 
Marker (GPM) 4W. The flight crew was cleared to 
taxi to Runway 18L via Taxiways A2, A, Y, and D 


(Figure 5, magenta route). As the ownship taxied on 
Taxiway A, an aircraft was taxiing toward the 
ownship on Taxiway A. Provided the flight crews of 
both aircraft followed the STBO guidance, the 
aircraft would collide at the intersection of Taxiways 
A and Y. 

Taxi Intersection Conflict Scenario 

In this scenario, both aircraft were given STBO 
taxi clearances that were conflict free, but the failure 
of the traffic aircraft to comply with the taxi 
clearance resulted in a taxiway intersection conflict. 


The same approach and departure flow as 
previous was used. 



Figure 6. Taxi Intersection Conflict Scenario 

At the beginning of this scenario, the ownship 
was parked on the ramp near GPM 4W. The flight 
crew was cleared to taxi to Runway 18L via 
Taxiways A2, A, B, S, and D (Figure 6, magenta 
route). As the ownship taxied on Taxiway A, an 
aircraft was taxiing on Taxiway S from GPM 10, 
toward Taxiway A. The aircraft was given clearance 
to taxi to the terminal via Taxiways S, A, and N 
(Figure 6, green route). However, the flight crew 
blunders and continues straight on Taxiway S instead 
of turning onto Taxiway A. The aircraft conflicts 
with the ownship at the intersection of Taxiways S 
and B, provided the flight crew followed their STBO 
guidance. 

Procedure 

Prior to the testing phase, each test subject 
participated in a briefing and training session. 


During the training for the CD&R capability, the 
flight crews were trained to stop if a warning alert 
was generated during taxi. They were not required to 
take evasive action when a caution alert was issued. 

Before each trial, the flight crew was briefed on 
the run conditions, e.g., departure runway, HUD 
utilization, and ATC radio frequencies. 

STBO taxi operations began in the ramp area 
with an expected taxi clearance received by the 
ownship via data-link. An example data-link 
expected taxi clearance is as follows: “EXPECT 
TAXI TO RWY 36C FROM SPOT 30 VIA C, E, J, 
R WITH EXPECTED TAXI RELEASE TIME 
12:32:00Z. MONITOR GROUND 121.9.” The 
expected taxi clearance was displayed as a dotted 
magenta line on the AMM. After acknowledging the 
expected taxi clearance by data-link, the flight crew 
taxied to the indicated GPM (also known as spot). 
After reaching the GPM, the actual taxi clearance 
was received via voice and data-link instructions. An 
example taxi clearance was: “TAXI RWY 36C BY 
12:37:30Z VIA L-S-R. CROSS RWY 36C, TAXI 
RELEASE TIME 12:32:00Z.” The taxi clearance was 
displayed as a solid magenta line on the AMM. The 
crew was required to respond to the taxi clearance via 
voice and data-li nk . The crew was to begin taxi as 
close as possible to the taxi release time and notify 
ATC they were commencing taxi. Once taxi began, 
the flight crew was requested to follow the taxi 
clearance and STBO guidance provided on the AMM 
and HUD, if utilized. The crews were instructed that 
acceptable performance was obtained if their RTA 
was met within +/- 15 seconds. The trial terminated 
once the aircraft stopped at the runway hold line. 

The test runs were documented via audio, video, 
and digital data recordings, and post-run and post-test 
questionnaires. 

Test Metrics 

For conflicts, a near-collision was counted if the 
center-of-gravity (CG) of the two aircraft were <185 
ft apart laterally. A collision was counted if the 
aircraft CGs were < 150 ft apart laterally. 


Results 

A summary of quantitative and qualitative 
results is presented. All data are referenced from the 
aircraft CG. The data was analyzed using parametric 
and non-parametric statistics, as appropriate, using an 
apriori level of significance of 0.05. 

STBO Taxi Conformance Results 

Conformance in following STBO taxi guidance 
was evaluated by comparing actual performance 
against planned guidance at various locations along 
the cleared STBO taxi route. The following locations 
were evaluated for conformance: route start, straight 
segment mid-points, entering and exiting 90 degree 
turns, and route end. These locations were chosen to 
ensure homogeneity across the scenarios for analysis 
purposes. The independent variables for this analysis 
were Map condition (Map C and Map D) and the 
absence or presence of a HUD showing STBO 
guidance information. The dependent measure was 
the conformance time in seconds which was 
displayed to the flight crew on the AMM as seconds 
early (+) or late (-). Histograms are also included that 
present the conformance time grouped into 5 second 
bins. 

STBO Route Start Location 

The crew was instructed to begin taxi as close as 
possible to the taxi release time. The “route start” 
conformance time measurement was taken when the 
aircraft CG crossed the hold line near the GPM. 

Data for all 72 STBO test trials but one (nominal 
and off-nominal) were analyzed for start location 
conformance. Data for one crew was considered an 
outlier and omitted because the crew held for traffic 
in the ramp area before taxiing to the designated 
GPM, causing them to miss the release time. In all 
other test runs, the crew taxied to the GPM prior to 
the release time and could begin the trial in a timely 
manner. 

After the taxi release time occurred, it took time 
for the engines to spool up and the aircraft to begin 
moving. As a result, for all test trials, the 
conformance measurement at the start location was 
always negative (Figure 7), i.e., late in comparison to 
the planned STBO guidance. The flight crews were 
within +/- 15 seconds of the planned STBO release 



time on 54.9% (39 of 71) of the trials. The 95 th 
percentile data was between 7.8 and 20 seconds late. 
No significant effects were found for Map condition, 
HUD, or Map by HUD interaction. 

Many pilots made comments regarding starting 
out with a late conformance time. As a result, some 
pilots started advancing the throttles slightly before 
the release time to begin engine spool up earlier on 
later trials. Future STBO designs need to incoiporate 
more latency in crew and aircraft spool-up at the start 
of the STBO route. 

40 



Figure 7. STBO Route Start Location 
Conformance 

STBO Straight Segment Mid-Point Locations 

For this measurement, the conformance time 
was taken near the mid-point of a straight taxi 
segment. Some cleared STBO taxi routes only had 
one straight segment while others had multiple 
straight segments. Data for all STBO test trials 
(nominal and off-nominal) were analyzed for the 
straight segment mid-point location (a total of 108 
data points). 

In general, the flight crews were able to conform 
well to the STBO guidance during the straight taxi 
segments (Figure 8). The crews were able to maintain 
+/- 1 0 seconds of the guidance at the straight segment 
mid-point location 86.1% (93 of 108) of the time and 
+/- 15 seconds for 92.6% (100 of 108) of the time. 

There was a significant main effect found for 
HUD condition. Pilots had significantly better STBO 
guidance conformance, on the straight route 
segments, using a HUD (mean, p, = 1.1 sec, standard 
deviation, a, = 11.3 sec) than without using a HUD 
(p = -2.6 sec, cj = 7.9 sec). 

In the straight-segments, crews were able to 
adjust their speed to meet their RTA. When viewed 
as conformance along the route, the data shows 


relatively good performance (92.6% compliance) but 
with some very significant tails in the data (i.e., one 
~50 seconds early and several ~30 seconds late). 
Along route conformance was suboptimal in these 
cases since the crews were only asked to meet the 
end-of-route RTA performance. Intermediate/ 
waypoint RTAs were not used. 

bO - 

4b - 
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Figure 8. STBO Straight Segment Location 
Conformance 

STBO 90 Degree Turn Locations 

To analyze STBO performance during 90 degree 
turns, conformance time was taken 20 seconds prior 
to the center of the turn and 20 seconds after the 
center of the turn based on the planned route 
guidance. Some cleared STBO taxi routes only had 
one 90 degree turn while others had multiple 90 
degree turns. Data for only nominal STBO test trials 
were analyzed because the off-nominal scenarios did 
not contain any 90 degree turns. 

Of the 96 data points available, only 90 were 
included in the analysis. On three trials using the 
Map D condition, a wrong turn was made and data 
were not available. (After discussing the error with 
the crews, it was determined that the Captains 
mistakenly thought that their aircraft was at the 
location of the trend symbol instead of the white 
chevron and, as a result, turned early.) For three other 
trials, one turn produced the outliers. The ownship 
was taxiing on Taxiway C for a turn onto Taxiway A. 
Another aircraft was taxiing along Taxiway A. On 
these three occasions, the ownship STBO 
conformance was late approaching the turn and the 
crew did not think the turn could be made onto 
Taxiway A in advance of the other aircraft; therefore, 
they held for the other traffic and thus, were very late 
entering the turn. Since performance was 
significantly different than on the other trials, these 
data were omitted as outliers. 


There was a wide variability for conformance at 
the entry and exit location of the 90 degree turns 
(Figures 9 and 10). The STBO guidance was reduced 
to 4 kts on most 90 degree turns. This is apparently 
slower than turns are usually taken because, in 
general, the actual speed during the turns (center of 
turn) was higher (p = 9.5 kts, a = 2.7 kts) than the 
advised speed (p = 4.5 kts, a = 1.4 kts) and, as a 
result, conformance time at the exit location was 
earlier (p = 9.5 sec, a = 13.6 sec) than conformance 
time at the entry location (p = 0.9 sec, a = 13.9 sec). 
The entry location was reached early 51.1% (46 of 
90) of the time, while the exit location was reached 
early 77.8% (70 of 90) of the time. As some of the 
pilots realized that the planned speed was very slow 
in the turns, they began going into them late so the 
turn could be taken faster. Many pilots commented 
that the STBO speed guidance in the turns was too 
slow and, for a transport category aircraft, should be 
8 to 10 kts. 
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Seconds 

Figure 9. STBO Enter 90 Degree Turn Location 
Conformance 

When entering a 90 degree turn, the flight crews 
maintained +/- 15 seconds of the planned STBO 
guidance on 78.9% (71 of 90) of the turns. No 
significant effects were found. 
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Figure 10. STBO Exit 90 Degree Turn Location 
Conformance 


When exiting a 90 degree turn, the flight crews 
maintained +/- 15 seconds of the planned STBO 
guidance on 73.3% (66 of 90) of the turns. No 
significant effects were found. 

STBO Route End Location 

The goal of providing STBO guidance was to 
enable flight crews to reach the end of the taxi route 
at the RTA. This “RTA point” was shown on the 
AMM as a green diamond near the end of the cleared 
route. The conformance time measurement was taken 
when the aircraft CG crossed the RTA point. Data for 
only nominal STBO test trials were analyzed because 
the RTA point was never reached during the off- 
nominal trials. 

Of the 48 data points available, only 47 were 
included in the analysis. On one trial, the crew 
slowed to conduct the before takeoff checklist just 
before crossing the RTA point, making the 
conformance time very late (-27 seconds). Since 
performance was significantly different than on the 
other trials, this data was omitted as an outlier. 

STBO route end location data is presented in 
Figure 11. On 97.9% (46 of 47) of the trials, the RTA 
point was crossed within +/- 10 seconds of the RTA, 
and within +/- 5 seconds of the RTA on 87.2% (41 of 
47) of trials. The crews were instructed that 
acceptable performance was met if the RTA point 
was reached within +/- 15 seconds of the RTA; these 
data show excellent compliance to this goal. 
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Seconds 


Figure 11. STBO Route End Location 
Conformance 

On the one trial in which the 15 second 
requirement was not met, the crew was focused on 
oncoming traffic near the taxi route and crossed the 
RTA point 19.9 seconds early. No significant effects 
were found. 


The STBO data shows that intermediate 
performance could vary significantly but the crews 
met their instructed goal - the end route RTA. If 
compliance along the route is necessary, intermediate 
RTAs/waypoints will be needed. The data also shows 
that STBO guidance development is not and may 
never be a trivial task. An STBO RTA significantly 
impacts air carrier Standard Operating Procedures 
(SOPs) and crew procedures (e.g., influencing when 
checklists can be started and completed), taxi speeds 
(including how this may vary by aircraft type and 
aircraft loading) and how they vary with traffic, taxi 
route (e.g., turns, hot spots) and prevailing visibility, 
and passenger comfort (e.g., how quickly to turn, 
accelerate, and decelerate). 

Off-Nominal Scenario Results 

As described in the Test Method section, the off- 
nominal trials were conducted between subjects to 
evaluate collision avoidance capability during STBO. 
The focus of the off-nominal scenarios was to 
determine the usefulness of traffic intent information 
for taxi collision avoidance and to determine the 
usefulness of CD&R for collision avoidance during 
STBO. Each crew evaluated the off-nominal 
scenarios using either the Map C or D condition with 
conflict traffic transmitting either NACp 9 or NACp 
10 position accuracy; therefore, three data points 
were collected for each combination of conditions. 
Intent information was displayed graphically on Map 
D but only textually on Map C. Alerts were not 
issued for traffic transmitting NACp 9 position 
accuracy, but were issued for traffic transmitting 
NACp 10 position accuracy. Since all traffic on the 
airport was transmitting position accuracy of NACp 9 
and higher, all traffic was displayed on the AMM. 

Taxi Head-On Conflict Scenario Results 

The taxi head-on scenario was given as Trial 3 
of six trials for each crew. This scenario simulated 
faulty SBTO guidance where conflicting traffic 
would meet the ownship head-on at an intersection, if 
the ownship followed the STBO guidance. 

For all of the taxi head-on trials, the ownship 
either slowed down or stopped for traffic to turn 
ahead of them at the intersection, averting a collision 
or conflict (Table 2). The flight crew was aware of 
the conflict traffic on each trial by either viewing the 
traffic intent information or viewing the traffic OTW. 


It was relatively easy to visually acquire the traffic 
OTW since the visibility was 1,800 ft. 

Alerts were possible on the conflict traffic on six 
taxi head-on trials (i.e., NACp 10). Nine caution 
alerts were generated for six trials prior to reaching 
the conflict intersection, at an average distance of 577 
ft from the traffic (a = 300 ft). The caution alert was 
generated earlier when the ownship was traveling 
faster. Multiple alerts were issued for two crews. (No 
provisions were made to reduce the occurrence of 
alert toggling which occurs when multiple instances 
of alerts are generated as a result of position accuracy 
or aircraft maneuvering. Alert toggling can be a 
distraction to the flight crew and could cause mistrust 
in the technology.) A warning alert was also 
generated on all three trials, 268 ft mean distance 
from the traffic (a = 15 ft). 

Three trials where traffic was transmitting 
NACp 10 accuracy (i.e., with alerts generated) 
resulted in near-collisions per our criteria. The near- 
collision calculation was made by determining 
whether the aircraft CGs were separated by less than 
185 ft laterally; which is different than the closest- 
point-of-approach (CPA) calculation (slant range). 
Two of the near-collisions occurred at the conflict 
intersection because the pilot slowed instead of 
stopping and eventually met the near-collision 
criteria. The third near-collision occurred when 
taxiing behind the conflict traffic after turning at the 
conflict intersection. The subject crew had gotten 
behind in following the STBO guidance while 
waiting for the conflict traffic to turn ahead, and was 
trying to reacquire the STBO planned timing but, 
because they were judging separation distance 
visually, met the near-collision criteria. 

After turning at the subject intersection, alerts 
were generated while following the taxiing conflict 
traffic. The pilots were trying to reacquire their 
STBO conformance and taxied close enough to the 
traffic to generate alerts. Four caution alerts were 
generated for three trials, at an average distance of 
280 ft from the traffic (a = 83 ft). One warning alert 
was generated on one trial 222 ft from the traffic. 

Alerts were not generated for any other traffic 
during these off-nominal trials. 



Table 2. Taxi Head-On Scenario Results 


Map 

NACp 

Action 

Outcome 

CPA (feet) 

Viewed Intent 

C 

9 

Stopped 


202.1 

Yes 

C 

9 

Slowed 


274.0 

Yes 

c 

9 

Stopped 


275.8 

Yes 

c 

10 

Slowed 


317.4 

Yes 

c 

10 

Slowed 

Near-collision 

213.4 

Yes 

c 

10 

Slowed 

Near-collision 

186.6 

Yes 

D 

9 

Slowed 


384.5 

Yes 

D 

9 

Stopped 


268.8 

Yes 

D 

9 

Stopped 


213.2 

Yes 

D 

10 

Slowed 

Near-collision 

222.2 

Yes 

D 

10 

Stopped 


333.6 

Yes 

D 

10 

Stopped 


319.3 

Yes 


Table 3. Taxi Intersection Scenario Results 


Map 

NACp 

Action 

Outcome 

CPA (feet) 

Viewed Intent 

C 

9 

Stopped 


303.9 

Yes 

C 

9 

Slowed 

Near-collision 

214.6 

Yes, late 

C 

9 

Stopped 

Near-collision 

251.7 

Yes 

C 

10 

Stopped 


262.9 

Yes 

C 

10 

Stopped 

Near-collision 

170.4 

No, stopped based on 
caution 

C 

10 

Stopped 


220.5 

Yes, stopped based on 
caution 

D 

9 

Turned in front 


36.2 

Yes 

D 

9 

Turned in front 


42.2 

Yes 

D 

9 

Stopped 

Near-collision 

179.7 

Yes, early 

D 

10 

Stopped 


303.0 

Yes 

D 

10 

Stopped 


268.9 

Yes 

D 

10 

Stopped 


235.4 

Yes, early 


Taxi Intersection Conflict Scenario Results 

The taxi intersection scenario was given as Trial 6 of 
six trials for each crew. This scenario simulated a 
blunder with the conflicting traffic missing its 
planned turn and instead, incurring from the right of 
the ownship, if the ownship followed the STBO 
guidance. 

One or both pilots were aware of the conflict 
traffic on each trial by either viewing the traffic 
symbol or traffic intent on the AMM or viewing 
OTW. For 10 of the taxi intersection trials, the 
ownship either slowed down or stopped for the 
taxiing traffic prior to the intersection (Table 3). On 
two trials, the pilots were early in following their 
STBO planned guidance (8 and 10 seconds) at the 
intersection, they were aware of the conflict traffic 
and they knew the aircraft missed its turn. The crews 


decided to speed up and turn in front of the traffic. 
The automated conflict traffic taxiing from behind 
overran the ownship causing the two collisions. 
Alerts were not generated for the conflict traffic on 
these two trials because the traffic was transmitting 
NACp 9 accuracy. 

Four trials resulted in near-collisions. Three of 
the near-collisions occurred at the conflict 
intersection. The fourth near-collision occurred when 
taxiing behind the conflict traffic after turning at the 
conflict intersection. The ownship taxied very close 
to the traffic near the end of the route, trying to cross 
the RTA point. Only one near-collision occurred on 
the trials in which alerts were generated. 

Alerts were possible on six of the taxi 
intersection trials (NACp 10). A caution alert was 
generated for each of the six trials prior to reaching 



the conflict intersection, at an average distance of 460 
ft from the traffic (o = 32 ft). A warning alert was 
also generated on four of the trials, 265 ft mean 
distance from the traffic (a = 7 ft). 

After turning at the subject intersection, alerts 
were generated while following the conflict traffic. 
The pilots were trying to reacquire their STBO 
conformance and taxied close enough to the traffic to 
generate alerts. Three caution alerts were generated 
for two trials, at an average distance of 410 ft from 
the traffic (c = 123 ft). Multiple alerts were issued for 
one crew. One warning alert was generated on one 
trial 290 ft from the traffic. 

Alerts were not generated for any other traffic 
during these off-nominal trials. 

Qualitative Results 

Post-run and post-test questionnaires were 
administered. Most of the questions were rated on a 
scale of 1 (“strongly disagree”, “low”, “not useful”) 
to 7 (“strongly agree”, “high”, “very useful”). 

STBO - The pilots were neutral in rating STBO 
for increasing workload compared to current surface 
operations (p = 4.2, a = 1.9); however, they did feel 
that STBO would increase their general situation 
awareness (p = 6.2, a = 1.2). They moderately agreed 
that STBO will increase efficiency (p = 5.0, a = 1.6) 
and safety (p = 5.0, a = 2.0) for ground movement of 
aircraft. Pilots generally commented that STBO 
negatively impacts workload and reduces the ability 
to watch for other traffic. Additional comments were 
STBO can be distracting; takes one pilot out of the 
loop; realistic taxi speed and parameters should be 
utilized as aircraft performance differs, large aircraft 
must consistently manage their energy and 
momentum; and STBO works in a “sterile” 
environment, taxi operations differ among fleets and 
airports. Concerns were expressed regarding how the 
STBO system will account for routing modifications, 
STBO guidance non-compliance, and taxi errors. 

STBO Symbology - Pilot ratings indicated that 
STBO would cause a moderate increase in head- 
down time compared to current-day operations (p = 
4.8, a = 2.0). Pilots commented that STBO caused 
more heads-down time than normal. The pilots felt 
that STBO information should be located on the 
AMM on the ND (18 pilots), HUD (15 pilots), and an 


AMM located on the EFB (14 pilots); however, they 
did not feel STBO information should be on the PFD 
(3 pilots) (pilots were requested to select all that were 
required). 

The Captains agreed that the advised ground 
speed information on the HUD was useful (p = 5.8, a 
= 1.8); however, they also felt that the advised 
ground speed alone, without any other STBO 
guidance information, was not optimal (p = 3.6, a = 

2.0) . Many pilots recommended also displaying the 
RTA time deviation (seconds early/late) on the HUD. 

Pilots that used Map Condition C indicated that 
it was easy to tell if they were going to reach the 
guidance end point on time, ahead of time, or behind 
schedule (p = 6.4, a = 0.9). All STBO symbology 
elements were rated as being useful, including the 
selected traffic information (p = 5.6, a = 1.6). 

Pilots that used Map Condition D also indicated 
that it was easy to tell if they were going to reach the 
guidance end point on time, ahead of time, or behind 
schedule (p = 6.0, a = 1.1). All STBO symbology 
elements were rated as being useful including the 
graphical traffic intent information (p = 6.5, a = 0. 7) 
and graphical STBO guidance (p = 5.8, a = 1.5). 
They felt that the graphical traffic intent information 
(traffic taxi route and guidance circles) was useful for 
determining the intent of aircraft (p = 6.0, a = 1.2). 
The pilots were neutral regarding the usefulness of 
the selected (textual) traffic information (p = 4.4, a = 

2.0) , thus, implying a preference for graphical 
information. However, the graphical symbology was 
also confusing to some as evidence by two crews 
misinterpreting the symbology. 

CD&R Symbology > and STBO - The pilots agreed 
that the CD&R system was effectively integrated 
with the STBO display concepts (p = 5.8, a = 1.0) 
and that CD&R was effective during STBO (p = 5.5, 
a = 1.1). The pilots slightly agreed that the traffic 
intent information was useful in identifying traffic 
conflict situations when using either map condition 
(Map C, p = 5.3, a= 1.8; Map D, p = 5.7, a = 1.9). 

Summary 

A piloted simulation study was conducted to 
evaluate the ability to safely conduct STBO by 
assessing the impact of providing traffic intent 



information, CD&R system capability, and the 
display of STBO guidance to the flight crew on both 
a head-down display and HUD. 

STBO was a new concept for the test subjects. 
On 98 percent of the trials, the flight crews met their 
RTA at the end of the taxi route within +/- 10 
seconds, well within the acceptable performance 
bounds of +/- 1 5 seconds of RTA. They also had very 
good STBO conformance during straight taxi 
segments, on 93% of the trials the crews were within 
+/- 15 seconds of the planned guidance. However, 
some of the design aspects of STBO (slow start-up at 
the release time, slow speeds in turns) were quickly 
identified by the crews and some of the pilots started 
advancing the throttles slightly before the taxi release 
time and ‘gaming’ the turns (i.e., going into them late 
so the turn could be taken faster). Conformance, 
therefore, varied widely at these locations. 

If compliance along the route is necessary, 
explicit intermediate RTAs/waypoints will be 
needed. Effective STBO will also need to account for 
SOPs, crew procedures, aircraft type and aircraft 
loadings, traffic, taxi route, prevailing visibility, and 
passenger comfort. 

The display of traffic intent information was 
found to be helpful, particularly to the first officer. 
Graphical presentation of intent was preferred over 
textual presentation; however, the method of 
presentation must be optimized in either case. The 
current graphical implementation was the cause of 
three wrong turns because the pilot mistakenly 
thought that their aircraft was at the location of the 
intent symbology. 

The CD&R system was minimally effective 
during STBO because the traffic was visible out-the- 
window (1,800 ft visibility). The flight crews were 
always aware of the traffic prior to any alerts being 
generated. Interestingly, two collisions occurred 
because the flight crews were early in reaching the 
intersection and turned in front of traffic; 
subsequently, the traffic taxied into the ownship. 
Alerts were not generated on these aircraft, so 
perhaps the crew was more willing to proceed ahead 
of the traffic. In some instances, alerts were issued 
when the ownship was following the traffic after 
turning at the intersection. 


Overall, the test subjects were neutral regarding 
the ability of STBO to increase efficiency. The pilots 
expressed concern for increased workload, the ability 
to watch for other traffic, and increased head-down 
time. Although not evaluated, pilots expressed 
concerns regarding how an STBO system will 
account for modifications to routing, STBO guidance 
non-compliance, and taxi errors. 
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